Ë Blockinóóð Ääääùù Ìðð× Èöóñóøø Ëó Blockin

S hool performan e tables" { an alphabeti al list of se ondary s hools along with aggregates of their pupils' performan e in national test { have been published in Great Britain sin e 1992. inevitably the media have responded by publishing ranked \league tables". Despite on ern over the potentially divisive e e t of su h tables the urrent Labour government has ontinued to publish this information in the same form. The e e t of this information on standards and the so ial makeup of the ommunity has been keenly debated. Sin e there is no ontrol group available that would allow us to investigate this issue dire tly, we present here a simple mathematial model. Our results suggest that in reasing so ial division is a natural onsequen e of publishing the information in this way. JAMES C. ROBINSON is a Royal So iety University Resear h Fellow in the Mathemati s Institute at the University of Warwi k. His primary resear h interests are in the rigorous mathemati al theory of the equations that model uid ows. REBECCA B. HOYLE is a Le turer in the Department of Mathemati s and Statisti s at the University of Surrey. She usually on entrates on mathemati al models related to the formation of patterns in a variety of di erent physi al systems. 2 INTRODUCTION The \Parent's Charter", rst published by the British Conservative government in 1991, promised the publi ation of examination results to allow parents to exer ise `informed' hoi e over their hildren's s hooling. A basis for national testing and the gathering of omparative data was provided by the Edu ation Reform A t of 1988 whi h established a uniform National Curri ulum. The resulting \S hool Performan e Tables" for se ondary s hools in England, S otland and Wales were rst published in 1992 (DfE 1992) under the then Se retary of State for Edu ation, John Patten. They take the form of an alphabeti al list of s hools with details of their publi examination results. Primary s hool tables followed in 1997 under Gillian Shephard, and are based on the performan e of 11 year olds in Key Stage 2 tests (DfEE 1996). The media qui kly seized upon this information, publishing \league tables" ranking s hools from best to worst a ording to their examination results. S hool league tables are not unique to the UK: standardised tests and league tables on the British model are now being piloted in Italy under the auspi es of the Istituto nazionale per la valutazione del sistema dell'istruzione (TES 2000); Australia publishes data on individual s hools' exam results; fortyve states in the US also publish `report ards' on s hools, and of those 27 rate s hools or identify low-performing s hools (Edu ation Week 2001). League tables are be oming in reasingly ommon beyond the s hool se tor as governments seek to provide greater information to the publi a ross a wide range of servi es. University and MBA league tables are now ommonpla e in the British and Ameri an press; in the UK this trend has a 3 elerated in response to the ready availability of `quantitative' data from the Resear h Assessment Exer ise, and Tea hing Quality Assessments. Hospitals and health providers are also ranked in tables published in both the UK and the US. In the UK the Patient's Charter led to the publi ation of High Level Indi ators and Clini al Indi ators for all National Health Servi e hospitals in June 1999 (NHS Exe utive 1999), and the introdu tion of ontroversial `star' ratings for hospitals in September 2001 (DofH 2001). Whilst the aim of bringing greater transparen y to our publi servi es is admirable, the publi ation of data in this raw form may indu e a feedba k e e t leading to deterioration of the very servi es the government is seeking to improve by its all to a ountability. We investigate this here in the parti ular ase of s hool performan e. S hool league tables have been seen widely as divisive and misleading, by both parents and tea hers, but nonetheless have been ontinued in the UK by the urrent Labour government. However, two of the regions with their own legislative assemblies devolved from entral Westminster ontrol hose to s rap the table earlier this year: rst Northern Ireland (where league tables were rst published in 1993) and then Wales. The opposition to league tables is based on the suggestion that their publi ation leads to the entren hment of di eren es in performan e between s hools as wealthier and more motivated parents make every e ort to ensure their hildren attend the \better" s hools. We investigate this further here. Due to the la k of a ontrol group1 (a problem ubiquitous throughout the so ial s ien es) it would be extremely diÆ ult to investigate the e e t of league tables on s hool standards and so ial makeup using observations taken \from life". Thus, in order to isolate these e e ts, we have onstru ted 1The abolition of league tables in Northern Ireland and Wales now provides the possibility of investigating a group where su h tables are not published. 4 a simple mathemati al model that we believe adequately des ribes the most important fa tors that in uen e students' exam performan e and the exer ise of parental hoi e based on published league tables. Perhaps unsurprisingly we observe the potentially divisive s enario outlined above, both in a very simple model of two s hools that we an treat analyti ally, and from a more sophisti ated model that we investigate numeri ally. From initially small random u tuations in exam results, a small group of s hools qui kly be ome established as \su essful", while a similar group rapidly start to \fail". As an alternative we suggest a rough qualitative banding of s hools into ex ellent, satisfa tory, and failing. This has the merit of redu ing the ampliation of small initial di eren es in performan e, whilst learly identifying both outstanding s hools to serve as models and problemati s hools in need of remedial measures. THE MATHEMATICAL MODEL In order to investigate the e e t of league tables on s hool numbers and s hool standards we need to model both the performan e of pupils and the movement of families. We onsider all the individual pupils in a single year group in a relatively isolated unit (e.g. a ounty), and al ulate the league tables for that year based on the average exam performan e at ea h s hool. In the next year we ll the s hools with new pupils from similar so ial ba kgrounds, having allowed for some movement of families in rea tion to the league tables. 5 Pupil performan e Sin e league tables are derived from exam results, our aim is to produ e a representative \exam result" based (potentially) on the following: the hild's innate ability, the in uen e of the hild's parents, and qualities that vary from s hool to s hool. Our rst task is to try to quantify the relative e e t of these di erent fa tors. In the 1970s Jen ks and o-workers on luded that at least half the variability in pupils' performan e ould be assigned to di eren es in their prior attainment and so ial ba kground. Of the remaining variation, it appears that relatively little an be dire tly attributed to the s hool itself: early studies (Aitkin & Longford, 1986) suggested a variation of only 2% dire tly attributable to s hool fa tors, although more re ent studies suggest a somewhat higher level of in uen e (e.g. around 10% in Willms 1987; 5{8% in Gray & Wil ox 1995, hapter 6). Be ause of this, we will not take into a ount any variability between s hools in our initial model (although we dis uss some adjustments to this in our on lusion). We also believe that the assumption that the s hools provide a \uniform level of servi e" is useful in the ontext of this study in order to try to isolate the e e t of league tables themselves on s hool performan e. The in uen e of parental ba kground on pupils' a ademi su ess is now well attested. Indeed, it appears (Gray & Wil ox, 1995, hapter 5) that the more detailed the information olle ted about individual pupils, the greater the eviden e for the importan e of a hild's parents for their s hool performan e. Resear h by the Nottinghamshire LEA that took into a ount the 6 number of professionals in the hild's household on rms the importan e of parental ba kground on a ademi a hievement (see Gray & Wil ox, 1995, p94). We have hosen to redu e \innate ability" and \parental in uen e" to one variable ea h, measuring \innate ability" by an IQ, and the \parental inuen e" by a single number. Be ause of the latter's orrelation with in ome, in what follows we refer to this fa tor as \parental in ome", but it should be understood that this is a shorthand only. In line with observations in Plewis & Goldstein (1998), that prior attainment is responsible for twi e as mu h variation in exam results as so ioe onomi fa tors, we have weighted the IQ roughly twi e as heavily as parental in ome in determining the nal exam s ore. This is still a somewhat arbitrary division, and negle ts the e e t of so ioe onomi fa tors on prior attainment, but we will argue in the on lusion that a di erent allo ation would not signi antly a e t our results. We take the average exam s ore per pupil roughly in line with the average GCSE s ore (with A = 8, A = 7, et .): approximately 33, with roughly 50% lying between 38 and 28 ( f. observations in DfES 2000). The jth pupil is assigned an IQ (whi h we denote by ej) at random from a normal distribution with mean 100 and standard deviation 20 (we write this as N (100; 20)) { this orresponds roughly with the national distribution. Their parental in ome pj is also hosen at random from aN (3; 1) distribution. Then their exam result xj is simply linearly dependent on these two fa tors, with xj = pj + ej + C; where = 3, = 0:3, and C = 9. The hoi e of onstants is onsistent with the relative weightings and values dis ussed above. League table positions 7 will be based on the average exam results x in ea h s hool, where x is given by x = p+ e+ C with p and e are the average parental in ome and IQ. If we were to allow a di erent \value added" fa tor v for ea h s hool then we would have xj = pj + ej + C + v: One simple model of this value added fa tor an be obtained by assuming that this e e t arises only from the re ruitment of more e e tive tea hers, who are in turn attra ted by able pupils with motivated parents. In this ase we would have v = a p+ b e, whi h would only a e t the s hool's average exam result by hanging the onstants and , x = ( + a) p + ( + b) e + C: This gives another justi ation (in addition to the investigation of league table e e ts independent of other onsiderations) for dis ounting fa tors intrinsi to di erent s hools. Movement of families In our model parental hoi e is available solely on the basis of moving into the at hment area of a parti ular s hool: most s hools lie in only one at hment area and the expe tation is still that one will attend the lo al s hool. The parents of those who are to attend s hool in the following year an onsider buying a house in the at hment area of their preferred s hool. Sin e moving home is an expensive business, and is a natural disin entive to lower in ome families, we have made the han e of a family moving proportional to 8 their \in ome" pj, with the onstant of proportionality (set in our model to 1=20). This also takes into a ount the possibility that so ial fa tors in uen e parents' per eption of the importan e of edu ation, suggested by various resear hers (e.g. Moore & Davenport, 1990; E hols et al., 1990). If the family wishes to move, they then hoose the s hool for their hild on the basis of the league tables. S hools are then assigned to everybody in a way that favours those living in the at hment area (see se tion 4 for a more detailed dis ussion of how s hools are hosen by parents and allo ated by our omputational \lo al authority"). ANALYSIS OF TWO SCHOOLS Before presenting our numeri al results, we onsider an even simpler model that we an treat analyti ally. We take only two s hools of 100 pupils ea h, and do not impose any eiling on the number of pupils at ea h s hool. In a further simpli ation we suppose that the family in omes in ea h at hment area take only the integer values 1, 2, 3, 4, and 5, with equal numbers initially assigned to ea h level. To simplify things further we assume that a xed fra tion p of families with in ome p hoose to move. We denote by N (j) the number of pupils at s hool j, and initially set both to 100. As above, the average exam performan e at ea h s hool is given by x(j) = p(j) + e(j) + C; where p(j) is the average parental in ome, and ea h e(j) is a random variable, e(j) N 100; 20 pN (j)! : We use the same parameters as in our simulations, = 3, = 0:3, C = 19 and = 0:05. 9 Whi h s hool performs better in the exams in a parti ular year is de ided at random depending on the pupils' IQs: the di eren e between exam marks (s hool one minus s hool two) is given by x = p+ "; where p = p(1) p(2) and " = ( e(1) e(2)) is a random variable, " N 0; 6s 200 N1N2! : One of the s hools s hool \A" will perform better in the rst year, whereupon some families at s hool`B" will hoose to move: pupil numbers in the se ond year at s hool A, broken down by parental in ome, will be p 1 2 3 4 5 number of pupils 21 22 23 24 25; 115 in total, with only 85 pupils at s hool B. This move gives an exam \advantage" of +0.615 to s hool A. The random u tuations arising from the pupils average IQ (re e ted in the variable ") now only gives s hool B a 25% han e of doing better than s hool A. Assuming the more likely out ome { that s hool A does better on e again { more families will swit h s hool at the end of the se ond year, with the numbers at s hool A now totally 128: p 1 2 3 4 5 number of pupils 22 24 26 27 29: S hool A now has an in reased exam advantage of +1.12 and s hool B is left with only an 11% han e of at hing up. Continuing in this way s hool A will very soon develop an unassailable lead as the more motivated families ontinue to move their hildren. Within a short time the rea tion to league tables has produ ed signi antly di erent 10 standards in the two s hools and raised the a uen e of one at hment area while depressing that of the other. NUMERICAL MODELLING The e e t of many parents a ting individually upon published information in sele ting s hools for their hildren is simulated by our numeri al ode. We model a situation where there are 10 s hools in a neighbourhood, and a total of 1000 pupils attending them. At the beginning of the simulation, ea h s hool has exa tly 100 pupils but, allowing a little variation in the numbers, we assign a maximum apa ity of 125 pupils to ea h s hool. Ea h pupil attends s hool for one iteration of the model, i.e. one s hool areer, whi h here we shall refer to as a s hool `year'. At the end of every `year' ea h hild attending s hool is repla ed by a hild from a similar so ioe onomi ba kground, su h as a younger sibling or a younger hild from the same neighbourhood. Before a new s hool year ea h s hool works out how many spare pla es it has to o er based on the number of pla es that were lled in the previous year. Ea h family has a probability pj that it will wish to move: those that do then move into the at hment area of the best s hool that has pla es available (these families are taken in a random order to prevent any unfair advantage). New va an ies may now be available, and a se ond round gives another han e to those families that were interested in moving but were unable to nd a better s hool with spare pla es in the rst round. Those who did not want to move { or who have been frustrated in both the rst and se ond rounds { remain at the same s hool as the previous year. 11 The entral issue here is how parents de ide whi h is the best s hool, and we onsider three possibilities in turn. Case 1: No published information In this simulation the parents hoose randomly, simulating the e e t of publishing no information. As an be seen in Figure 1, pupil numbers remain roughly onstant over time, while exam results show random u tuations. Case 2: League tables S hools are ranked a ording to their exam results in the previous year. When parents base their de isions on this ranking several s hools show a lear trend in both pupil numbers and exam results (Figure 2), the pupils following the results with a time lag of one year. The s hools that initially have good examination results, as a result of random u tuations about the mean, in rease their advantage over time, attra ting more pupils from a uent families and thereby maintaining a di erential in exam performan e over neighbouring s hools. Equally, s hools that start out with poor exam results rapidly lose their more a uent pupils and exam performan e su ers as a result. Random u tuations in pupil ability an break the trends in exam results from time to time, but the trends in pupil numbers are mu h more entren hed, as families prefer to stay in their own neighbourhood if possible. Case 3: Banded tables Here s hools are divided into three bands, rated `ex ellent' if their average exam result is above 106% of the average for all s hools, `failing' if their average result is below 94% of the average, and `satisfa tory' otherwise. In the rst simulation (Figure 3) s hool number 10 is graded `failing' in year 2, as is s hool 5 in year 9 and s hool 7 in years 6 and 7; all re over to `satisfa tory' 12 level in the next year. S hool number 1 is graded `ex ellent' in year 5, but afterwards falls ba k to `satisfa tory', while s hool number 2 is established as `ex ellent' in years 9 and 10. All other grades are `satisfa tory'. The number of pupils rises markedly at `ex ellent' s hools and equally drops substantially at `failing' s hools. The di erentials in pupil numbers are maintained mu h longer than the high or low gradings owing to the natural relu tan e of families to move: s hools lo k in their temporary su ess or failure over the longer term. To a ertain extent s hools also maintain their relative advantage or disadvantage in exam results: s hools 1 and 2 ontinue to do parti ularly well (but obviously not well enough to maintain their grading) in the years after their `ex ellent' rating, although it is interesting to note that s hool 7 rapidly re overs from its bad grading. Pupil numbers at the remaining s hools, graded `satisfa tory', adjust slightly to a ommodate the hanges at the `ex ellent' and `failing' s hools. Exam results in the `satisfa tory' s hools do not show lear trends. Banding appears to produ e less variation in exam results between s hools, but still auses migration of pupils, although to a lesser degree than the publi ation of league tables, and these hanges in population are then lo ked in by the preferen e for families to stay put. In a se ond simulation with stri ter rules to determine the two ex eptional bands (`ex ellent' when above 107% of the average and `failing' when below 93%) all s hools are graded `satisfa tory' for all years. As an be seen in Figure 4, pupil numbers remain onstant over time, while exam results show random u tuations. 13 CONCLUSION Our mathemati al model of parental hoi e, informed by the publi ation of exam-based league tables, shows that initial random u tuations in exam results are ampli ed, with pupils migrating from s hools lower down the table to higher-ranking s hools. A uent families are best pla ed e onomi ally to move their hildren, and at the same time tend to produ e better exam results. This feedba k e e t in reases the di erential between the best and worst performing s hools. Random u tuations in pupil ability an break the trend in exam results at any given s hool from time to time, but it be omes in reasingly diÆ ult for a s hool to improve its league table position as the gap in exam performan e between itself and the s hool above it widens. In our simple analyti al two-s hool model we have shown that it rapidly be omes almost impossible for the worse-performing s hool to at h up with the better-performing s hool by means of these random u tuations. We have also onsidered an alternative format for published information, where s hools are divided into bands (`ex ellent', `satisfa tory' and `failing') a ording to their exam performan e. There is a tenden y for pupils to migrate from s hools in lower bands to s hools in higher bands, and for exam results to improve at s hools in higher bands and worsen at s hools in lower bands in onsequen e. However, the migration and exam result distortion is less than in the ase where league tables are used, as fewer s hools are di erentiated from the bulk. If there is suÆ ient random variation owing to variations in pupil ability, the `ex ellent' and `failing' grades are not maintained over long periods of time, and so these di erentials between s hools are somewhat less persistent. The banding format does not require su h detailed testing as is urrently pra tised, and we believe ould be arried out as part of a standard OFSTED 14 s hool inspe tion. This would also allow more easily for adjustments to bemade to a s hool's banding by taking into a ount the so ial and e onomiba kground of the s hool and its pupils.The relative weightings of parental in uen e and pupils' innate abilityre e t the relative magnitudes of deterministi and random e e ts in themodel. Giving more weight to parental in uen e will result in more en-tren hed trends in exam performan e and pupil numbers, whereas weightingpupils' IQs more strongly will in rease the han es that random e e ts willallow s hools to es ape their trends. Here we have taken a generous viewof prior attainment, attributing it entirely to pupils' native wit, and hen ein line with Plewis & Goldstein (1998) have weighted IQ twi e as heavilyas parental in uen e; more realisti ally we expe t that so ioe onomi fa -tors will determine prior attainment to some degree. This would lead us toweight deterministi fa tors more strongly than we have done, and thus toeven stronger trends in pupil numbers and exam performan e, with mu hless han e of random u tuations allowing s hools to es ape the prevailingso ioe onomi onditions.In summary we have shown that the publi ation of s hool performan edata in the form of league tables is likely to lead to so ial division and awidening of the gap in attainment between the best and worst performings hools, even when those s hools provide identi al added value to their pupils.15 REFERENCESAITKIN, M. & LONGFORD, N. (1986) Statisti al issues in s hool e e tive-ness, Journal of the Royal Statisti al So iety A, 149, pp. 1{42.Department for Edu ation (1992) S hool Performan e Tables, PubliExamination Results 1992 (London, HMSO).Department for Edu ation and Employment (1996) Primary S hool Per-forman e Tables 1996, Key Stage 2 Results (London, HMSO).Department for Edu ation and Skills (2000) 2000 GCSE/GNVQ Addi-tional Ben hmark Information (London, HMSO).Department of Health (2001) NHS Performan e Ratings A ute Trusts2001/01 (London, HMSO).Edu ation Reform A t 1988 (London, HMSO).ECHOLS, F., M PHERSON, A., & WILLMS, J.D. (1990) Parental hoi ein S otland, Journal of Edu ation Poli y, 5, pp. 207{222.Edu ation Week (2001) Quality Counts 2001: A Better Balan e, Editorialproje ts in Edu ation, 20, no. 17.GRAY, J. & WILCOX, B. (1995) Good S hool, Bad S hool (Open Uni-versity Press, Bu kingham).JENCKS, C. ET AL. (1972) Inequality: A Reassessment of the e e tsof family and s hooling in Ameri a (Basi Books, New York).MOORE, D. & DAVENPORT, S. (1990) Choi e: the new imporved sortingma hine, in: W.L. BOYD & H.J. HALBERG (Eds) Choi e in Edu ation:Potential and Problems (M Cut han, Berkeley, CA).16 NHS Exe utive (1999) Quality and performan e in the NHS: HighLevel Indi ators (London, HMSO).NHS Exe utive (1999) Quality and performan e in the NHS: Clini alIndi ators (London, HMSO).PLEWIS, I. & GOLDSTEIN, H. (1998) Ex ellen e in S hools: a failure ofstandards, British Journal of Curri ulum and Assessment, 8, pp. 17{20.Times Edu ational Supplement (2000) Italians follow UK lead on testingWILLMS, J.D. (1987) Di eren es between S ottish Edu ationa Authori-ties in their examination attainment, Oxford Review of Edu ation, 13,pp. 211{237.17 CAPTIONS FOR FIGURESFigure 1. Random hoi e simulation: a) The exam results and b) the pupilnumbers for ten s hools shown over ten years where parents hoose s hoolsrandomly.Figure 2. League table simulation: a) The exam results and b) the pupilnumbers for ten s hools shown over ten years where parents hoose s hoolsa ording to a published league table based on the previous year's examina-tion results.Figure 3. Banded table simulation 1: a) The exam results and b) the pupilnumbers for ten s hools shown over ten years where parents hoose s hoolsa ording to a published bands based on the previous year's examinationresults. The band gradings for ea h s hool are given in the text.Figure 4. Banded table simulation 2: a) The exam results and b) the pupilnumbers for ten s hools shown over ten years where parents hoose s hoolsa ording to a published bands based on the previous year's examinationresults. Here all s hools are graded `satisfa tory' for all years.18 Figure 1.1 2 3 4 5 6 7 8 9 102530schoola) timeaverageexamresults1 2 3 4 5 6 7 8 9 10050100schoolb) timenumberofpupils19 Figure 2.1 2 3 4 5 6 7 8 9 102530schoola) timeaverageexamresults1 2 3 4 5 6 7 8 9 10050100schoolb) timenumberofpupils20 Figure 3.1 2 3 4 5 6 7 8 9 102530schoola) timeaverageexamresults1 2 3 4 5 6 7 8 9 10050100schoolb) timenumberofpupils21 Figure 4.1 2 3 4 5 6 7 8 9 102530schoola) timeaverageexamresults1 2 3 4 5 6 7 8 9 10050100schoolb) timenumberofpupils22